LDA

We will compare some classifiers on the “Toxic” column.

Load libraries

library(tidyverse)

package 㤼㸱tidyverse㤼㸲 was built under R version 4.0.5Registered S3 methods overwritten by 'dbplyr':
  method         from
  print.tbl_lazy     
  print.tbl_sql      
-- Attaching packages --------------------------------------------------------------------------------------------------------------------------- tidyverse 1.3.1 --
v ggplot2 3.3.5     v purrr   0.3.4
v tibble  3.1.3     v dplyr   1.0.7
v tidyr   1.1.3     v stringr 1.4.0
v readr   2.0.1     v forcats 0.5.1
package 㤼㸱ggplot2㤼㸲 was built under R version 4.0.5package 㤼㸱tibble㤼㸲 was built under R version 4.0.5package 㤼㸱tidyr㤼㸲 was built under R version 4.0.5package 㤼㸱purrr㤼㸲 was built under R version 4.0.5package 㤼㸱dplyr㤼㸲 was built under R version 4.0.5package 㤼㸱stringr㤼㸲 was built under R version 4.0.5package 㤼㸱forcats㤼㸲 was built under R version 4.0.5-- Conflicts ------------------------------------------------------------------------------------------------------------------------------ tidyverse_conflicts() --
x dplyr::filter() masks stats::filter()
x dplyr::lag()    masks stats::lag()

library(tictoc)

package 㤼㸱tictoc㤼㸲 was built under R version 4.0.5

library(caret)

package 㤼㸱caret㤼㸲 was built under R version 4.0.5Loading required package: lattice
Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     

Attaching package: 㤼㸱caret㤼㸲

The following object is masked from 㤼㸱package:purrr㤼㸲:

    lift

library(MASS)

Attaching package: 㤼㸱MASS㤼㸲

The following object is masked from 㤼㸱package:dplyr㤼㸲:

    select

source("./parameters.R")

Open the Bag of Word with labels

# Open the bag of words
fileName = "bow_tfidf__min_words_100_2grams_1000__sampling_balanced__cor_cut_0.5_from_1408_to_1247_rm0.csv"
df = read_csv(fileName, col_types=col_types_df)
df = df[,-c(2,3,5:9)]
# During tests, we can work on a sample
sampled = FALSE
if (sampled == TRUE) {
  set.seed(42)
  max = nrow(df)
  sampled = round(max/10)
  df = df[sample(max, sampled), ]
}
# show the data set
df

Splitting the data

# Split between train and test
df_train = df[df[1] == 1,-1]
df_test  = df[df[1] == 2,-1]

# Split the test set between features and labels
X_train = df_train[,-1]
Y_train = as.factor(df_train$df_toxic)

# Split the train set between features and labels
X_test = df_test[,-1]
Y_test = as.factor(df_test$df_toxic)

Have a look

X_train

as.data.frame(Y_train) 

X_test

as.data.frame(Y_test)

Train the model

tic("Training: ")

f <- lda(X_train,Y_train)

toc(log = TRUE)

Training: : 227.22 sec elapsed

This takes some time…

What’s in f ?

Restarting R session...

Let’s do the inference on the test set now.

# Do the inference on the test set
tic("Inference: ")
predictions = predict(f,X_test)
toc(log = TRUE)

Inference: : 5.75 sec elapsed

Y_pred = as.data.frame(predictions)
Y_pred

What does the confusion matrix gives us?

writeLines("\n")

mat = confusionMatrix(Y_test, as.factor(Y_pred$class))
mat

Confusion Matrix and Statistics

          Reference
Prediction    0    1
         0 3714 1916
         1 1112 4643
                                          
               Accuracy : 0.734           
                 95% CI : (0.7258, 0.7421)
    No Information Rate : 0.5761          
    P-Value [Acc > NIR] : < 2.2e-16       
                                          
                  Kappa : 0.4672          
                                          
 Mcnemar's Test P-Value : < 2.2e-16       
                                          
            Sensitivity : 0.7696          
            Specificity : 0.7079          
         Pos Pred Value : 0.6597          
         Neg Pred Value : 0.8068          
             Prevalence : 0.4239          
         Detection Rate : 0.3262          
   Detection Prevalence : 0.4945          
      Balanced Accuracy : 0.7387          
                                          
       'Positive' Class : 0               
                                          

print("END: all the notebook ran.")

[1] "END: all the notebook ran."

Sys.time()

[1] "2021-08-24 13:31:48 CEST"

writeLines(paste0("File: ", fileName))

File: bow_tfidf__min_words_100_2grams_1000__sampling_balanced__cor_cut_0.5_from_1408_to_1247_rm0.csv

writeLines(paste0("Accuracy: ", mat$overall[1]))

Accuracy: 0.734036012296882

writeLines(paste0(tic.log(format = TRUE)[1][1]))

Training: : 179.6 sec elapsed

writeLines(paste0(tic.log(format = TRUE)[2][1]))

Inference: : 6.01 sec elapsed

Results

bow_tfidf__min_words_100_2grams_1000__sampling_balanced__cor_cut_0.3_from_1408_to_1110_rm0.csv

Accuracy: 0.735107618111994 Training: : 0.1 sec elapsed Inference: : 0.11 sec elapsed

Reference Prediction 0 1 0 3719 1847 1 1119 4512

bow_tfidf__min_words_100_2grams_1000__sampling_balanced__cor_cut_0.3_from_1408_to_1110.csv

Accuracy: 0.725739848589126 Training: : 179.6 sec elapsed Inference: : 6.01 sec elapsed

Reference Prediction 0 1 0 3708 2075 1 1113 4728

bow_tfidf__min_words_100_2grams_1000__sampling_balanced__cor_cut_0.5_from_1408_to_1247_rm0.csv

Accuracy: 0.734036012296882 Training: : 179.6 sec elapsed Inference: : 6.01 sec elapsed

Reference Prediction 0 1 0 3714 1916 1 1112 4643